Goods order processing method and apparatus, server, shopping terminal, and system

ABSTRACT

At least one associated image uploaded by a client is received, where the at least one associated image is obtained in response to detecting, by the client based on an image recognition method, an occurrence of a change of goods in a storage container. Based on the at least one image, difference information pertaining to a difference in the goods in the storage container is identified. Corresponding goods order change information is generated using the difference information. User order information corresponding to the client based on the goods order change information is updated. Updated user order information is sent to the client.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of PCT Application No.PCT/CN2018/107028, filed on Sep. 21, 2018, which claims priority toChinese Patent Application No. 201710873473.6, filed on Sep. 25, 2017,and each application is hereby incorporated by reference in itsentirety.

TECHNICAL FIELD

Implementations of the present specification belong to the field ofcomputer data processing technologies, and in particular, relate to agoods order processing method, apparatus, and system, a server, and ashopping terminal.

BACKGROUND

Fast development of computer and Internet technologies has boosted rapidrising of many emerging industries, and changed ways of shopping andprovided convenience for many consumers. A self-service supermarket is acurrent market expansion hotspot that major shopping platform operatorsare focusing on, and involve technical issues about how to convenientlyand reliably process order data of users.

At present, self-service supermarkets can adopt a self-service ordersettlement solution, A plurality of sensor devices such as cameras,pressure sensors, infrared sensors, and wireless network devices aredeployed in places like shelves and aisles in the malls to monitorcommodities taken out by customers from the shelves and complete ordersettlement in combination with devices used for self-service QRcode-based scanning payment, shopping cart-based weighing, and so on.Such a method needs a plurality of types of auxiliary devices, and alarge number of apparatuses need to be deployed in the malls, resultingin relatively high hardware costs for service implementation. Moreover,the method needs data collaboration of the sensor devices to collectstatistics of the commodities, and it is hard to ensure stability, whichis not conducive to large-scale promotion and application. Therefore,currently, a simple, convenient, and reliable order processing solutionwith lower costs is needed for self-service supermarkets.

SUMMARY

Implementations of the present specification aim to provide a goodsorder processing method, apparatus, and system, a server, and a shoppingterminal, so that a convenient, simple, and reliable order settlementmethod with lower implementation costs can be provided to reduceimplementation costs of a self-service supermarket automatic orderprocessing solution and improve order data processing reliability.

The goods order processing method, apparatus, and system, the server,and the shopping terminal provided in the implementations of the presentspecification are implemented in the following ways:

A goods order processing method is provided, where the method includesthe following: detecting, by a client based on an image recognitionmethod, whether a change occurs to goods in a storage container; if achange of the goods is detected, sending, by the client, an associatedimage of the change of the goods in the storage container to a server;identifying, by the server, information about the goods difference inthe storage container based on the associated image; generating, by theserver, corresponding goods order change information by using thedifference information; updating, by the server, user order informationcorresponding to the client based on the goods order change information;sending, by the server, updated user order information to the client;and displaying, by the client, the updated user order information.

A goods order processing method is provided, where the method includesthe following: receiving an associated image uploaded by a client, wherethe associated image includes an image obtained when the client detects,based on an image recognition method, that a change of the goods occursto goods in a storage container; identifying information about the goodsdifference in the storage container based on the associated image;generating corresponding goods order change information by using thedifference information; updating user order information corresponding tothe client based on the goods order change information; and sendingupdated user order information to the client.

A goods order processing method is provided, where the method includesthe following: detecting, based on an image recognition method, whethera change of the goods occurs in a storage container; if a change of thegoods is detected, sending an associated image of the change of thegoods in the storage container to a server; receiving updated user orderinformation returned by the server; and displaying the updated userorder information.

A goods order processing apparatus is provided, where the apparatusincludes the following: an image receiving module, configured to receivean associated image uploaded by a client, where the associated imageincludes an image obtained when the client detects, based on an imagerecognition method, that a change of the goods occurs in a storagecontainer; an image recognition module, configured to identifyinformation about the goods difference in the storage container based onthe associated image; an order generation module, configured to generatecorresponding goods order change information by using the differenceinformation; an order update module, configured to update user orderinformation corresponding to the client based on the goods order changeinformation; and an information feedback module, configured to sendupdated user order information to the client.

A goods order processing apparatus is provided, where the apparatusincludes the following: a change of goods detection module, configuredto detect, based on an image recognition method, whether a change of thegoods occurs in a storage container; an image sending module, configuredto send an associated image of the change of the goods in the storagecontainer to a server if a change is detected; an information receivingmodule, configured to receive updated user order information returned bythe server; and a display module, configured to display the updated userorder information.

A server is provided, including at least one processor and a memoryconfigured to store an instruction executable by the processor, wherewhen executing the instruction, the processor implements the following:receiving an associated image uploaded by a client, where the associatedimage includes an image obtained when the client detects, based on animage recognition method, that a change of the goods occurs in a storagecontainer; identifying information about the goods difference in thestorage container based on the associated image; generatingcorresponding goods order change information by using the differenceinformation; updating user order information corresponding to the clientbased on the goods order change information; and sending updated userorder information to the client.

A shopping terminal is provided, including at least one processor and amemory configured to store an instruction executable by the processor,where when executing the instruction, the processor implements thefollowing: detecting, based on an image recognition method, whether achange of the goods occurs in a storage container; if a change of thegoods is detected, sending an associated image of the change of thegoods in the storage container to a server; receiving updated user orderinformation returned by the server; and displaying the updated userorder information.

A goods order processing system is provided, including a shoppingterminal and a server, where the server includes any apparatus in theimplementations of the present specification; and the shopping terminalincludes any shopping terminal in the implementations of the presentspecification; or the server implements steps of any method in theimplementations of the present specification; and the shopping terminalimplements steps of any method in the implementations of the presentspecification.

According to the goods order processing method, apparatus, and system,the server, and the shopping terminal that are provided in one or moreimplementations of the present specification, a photographing device canbe installed in a shopping basket, a shopping cart, or other storagecontainers, and a remote server can identify and determine a user orderchange by monitoring a change of the goods in the storage container inreal time, and can generate or update a shopping list of a user. Assuch, end-user devices such as an intelligent shopping cart can be usedto process data of self-service supermarket orders in combination with aserver for goods identification and order processing to reduceself-service cashier devices, sensor devices, etc. With theimplementation solutions provided in the present specification, an imagecapture device or a video shooting device of a client can be used tosend image data of goods detection to a server, so that the serverperforms processes such as goods identification and order settlement.Therefore, overall hardware implementation costs can be lower. Inaddition, data exchanging nodes can be reduced and solutionimplementations can be simpler and more convenient. In actualapplications, implementation costs of the technical solutions forautomatic settlement processing of self-service supermarket goods orderscan be greatly reduced, and the technical solutions are convenient toimplement and have better stability. An implementation environment andimplementation factors such as self-service supermarket size, customerflow volume, and consumer behavior feature have relatively small impact,and facilitate large-scale promotion and use.

BRIEF DESCRIPTION OF DRAWINGS

To describe the technical solutions in the implementations of thepresent specification or in the existing technology more clearly, thefollowing briefly introduces the accompanying drawings for describingthe implementations or the existing technology. Apparently, theaccompanying drawings in the following description merely show someimplementations of the present specification, and a person of ordinaryskill in the art can still derive other drawings from these accompanyingdrawings without creative efforts.

FIG. 1 is a schematic flowchart illustrating an implementation of agoods order processing method, according to an implementation of thepresent specification;

FIG. 2 is a schematic diagram illustrating an implementation scenario ofphotographing a goods in a shopping cart by a client, according to oneimplementation of the present specification;

FIG. 3 is a schematic diagram illustrating an implementation scenario ofphotographing an added goods in a shopping cart by a client, accordingto one implementation of the present specification;

FIG. 4 is a schematic diagram illustrating identification of goodsboundaries for a change of the goods, according to one implementation ofthe present specification;

FIG. 5 shows an image within identified goods boundaries in animplementation scenario, according to one implementation of the presentspecification;

FIG. 6 is a schematic diagram illustrating a scenario that another goodsis stacked on the first goods in the shopping cart shown in FIG. 3;

FIG. 7 shows an image within goods boundaries, of the new goods in FIG.6, identified based on an image difference;

FIG. 8 is a schematic flowchart illustrating another implementation ofthe method, according to one implementation of the presentspecification;

FIG. 9 is a schematic diagram illustrating an implementation scenario ofa goods order processing method, according to one implementation of thepresent specification;

FIG. 10 is a schematic flowchart illustrating an implementation of themethod available on a server side, according to one implementation ofthe present specification;

FIG. 11 is a schematic flowchart illustrating an implementation of themethod available on a client side, according to one implementation ofthe present specification;

FIG. 12 is a schematic flowchart illustrating another implementation ofthe method available on a client side, according to one implementationof the present specification;

FIG. 13 is a schematic structural diagram illustrating modules of animplementation of a goods order processing apparatus available on aserver side, according to one implementation of the presentspecification;

FIG. 14 is a schematic structural diagram illustrating modules of animplementation of an image recognition module in the apparatus,according to one implementation of the present specification;

FIG. 15 is a schematic structural diagram illustrating modules ofanother implementation of the apparatus, according to one implementationof the present specification;

FIG. 16 is a schematic structural diagram illustrating modules ofanother implementation of the apparatus, according to one implementationof the present specification;

FIG. 17 is a schematic structural diagram illustrating modules ofanother implementation of the apparatus, according to one implementationof the present specification;

FIG. 18 is a schematic structural diagram illustrating modules of animplementation of a goods order processing apparatus, according to oneimplementation of the present specification;

FIG. 19 is a schematic structural diagram illustrating modules of animplementation of a change of the goods detection module in theapparatus, according to one implementation of the present specification;

FIG. 20 is a schematic structural diagram illustrating modules ofanother implementation of a change of the goods detection module in theapparatus, according to one implementation of the present specification;

FIG. 21 is a schematic structural diagram illustrating modules ofanother implementation of the apparatus, according to one implementationof the present specification;

FIG. 22 is a schematic structural diagram illustrating an implementationof the apparatus implemented in combination with computer hardware,according to one implementation of the present specification; and

FIG. 23 is a schematic architectural diagram illustrating animplementation of a system, according to one implementation of thepresent specification.

DESCRIPTION OF IMPLEMENTATIONS

To make a person skilled in the art understand the technical solutionsin the present specification better, the following clearly describes thetechnical solutions in one or more implementations of the presentspecification with reference to the accompanying drawings in the one ormore implementations of the present specification. Apparently, thedescribed implementations are merely some but not all of theimplementations of the present specification. All other implementationsobtained by a person of ordinary skill in the art based on the one ormore implementations of the present specification without creativeefforts shall fall within the protection scope of the solutions in theimplementations of the present specification.

Implementation solutions provided in the present specification can beapplied to, but not limited to, application scenarios of self-servicesupermarkets, and can adopt a Client/Server (C/S) system architecture.Generally, devices having a photographing or video recording function,such as smartphones, video cameras, monitors, and digital cameras, canbe installed in a plurality of types of goods loading devices (which canbe collectively referred to as storage containers in someimplementations of the present specification for ease of universaldescription), such as handheld shopping baskets, hand-pulled shoppingbaskets, shopping carts, and platform trucks that are provided inself-service supermarkets. These photographing or video recordingdevices can contain image change detection logic, and can detect changeof the goods status in a storage container based on a captured image ora video image. A client can send an image of a change of the goods to aserver side for processing. The server side can identify, based on theimage, a goods that is added by a user to the storage container or agoods that is taken out by the user from the storage container, andgenerate corresponding change of the goods information. Further, totalorder information of the user can be generated or updated based on theinformation about the change of the goods in the storage container. Theupdated total order information of commodities can be immediatelydisplayed on a client display device for the user to view, confirm, etc.

Notably, the C/S can be a logically divided client/server architecture.To be specific, in some implementations of the present specification,the client can include an end-user device that collects a goods image ofthe storage container, and the server can include an end-user devicethat obtains the image collected by the client and performs goodsidentification and order processing based on the image. In someimplementation products, the implementation solutions of the presentspecification can also be applied to an implementation scenario that theclient and the server are the same physical device. For example, adevice on a smart shopping cart can include an image change detectionmodule, and can also include an in-image-goods identification moduleand/or an order data processing module. Each smart shopping cart canperform self-service change of the goods detection, goodsidentification, order processing, etc. Order data of all smart shoppingcarts can be summarized to a specified order service server.

The following describes in detail one or more implementation solutionsof the present specification by using an application scenario that orderprocessing is performed on commodities in a user's shopping cart in aself-service supermarket. Notably, the implementation solutions of thepresent specification are not limited to implementation applications ina self-service supermarket scenario. The implementation solutionsprovided in the implementations of the present specification can also beapplied to other implementation scenarios, for example, animplementation scenario that a change of the goods in a storagecontainer is identified in a platform truck, a container, a truckcompartment, an express packing box, and so on and corresponding goodsorder data is processed. The self-service supermarket in theimplementations can include no or a few on-site cashiers that introducecommodities in normal shopping carts. The supermarket can include aconvenience store, a large shopping mall, a self-service station, etc. Auser can collect purchased commodities by using a shopping cart in thesupermarket, a photographing or video shooting device on the shoppingcart can detect a change of the goods in the shopping cart, andsubsequently a server can identify a goods and generate a correspondingorder. After the user completes a purchase, the user's shopping list canbe generated automatically, more rapidly, more reliably, and morestably. The user can voluntarily perform order settlement, for example,perform online settlement on a payment platform. After the ordersettlement succeeds, the user can pass exit or aisle without lining up,As such, user shopping experience can be greatly improved.

FIG. 1 is a schematic flowchart illustrating an implementation of agoods order processing method, according to one implementation of thepresent specification. Although the present specification providesmethod operation steps or apparatus structures shown in the followingimplementations or the accompanying drawings, the method or apparatuscan include more or fewer (after partially combined) operation steps ormodule units based on conventional or non-creative efforts. In a step orstructure that a necessary cause and effect relationship does notlogically exist, sequence of executing these steps or a module structureof the apparatus is not limited to execution sequence or a modulestructure shown in the implementations or the accompanying drawings ofthe present specification. In an actual apparatus, server, or end-userdevice product application, the method or the module structure can beperformed sequentially or in parallel based on method or modulestructure shown in the implementations or the accompanying drawings (forexample, in a parallel processor or a multi-threaded processingenvironment, or even an implementation environment including distributedprocessing and server clustering). A specific implementation is shown inFIG. 1. In an implementation of a goods order processing method providedin the present specification, the method can include the followingsteps.

S0: A client detects, based on an image recognition method, whether achange of the goods occurs in a storage container.

In the present implementation, the client can include an electronicdevice that is installed on a shopping cart and has an image changedetection function, such as a smartphone, a video camera, or a monitor.The client can further have a communication capability, and can send acaptured image or a shot video to a server. In an implementation method,the client can be independent apparatuses such as an end-user deviceinstalled on the storage container for image capture and an end-userdevice for image change detection. For example, the client can include asmartphone for photographing and video uploading and a dedicated devicefor image change detection. In other implementation methods, the clientcan alternatively include one end-user device that is composed of aplurality of devices such as an image capture apparatus, an image changedetection apparatus, and a communication module, or can even be combinedwith a storage container such as a shopping cart to form a whole of theclient. The storage container can include a plurality of types ofstorage apparatuses used for storing commodities purchased by users, forexample, a handheld shopping basket and a hand-pushed shopping cart.

In a specific example of the present implementation scenario, FIG. 2 isa schematic diagram illustrating an implementation scenario ofphotographing a goods in a shopping cart by a client, according to oneimplementation of the present specification. The storage container canbe a shopping cart used in a supermarket, for example, common shoppingcarts with various containers of 60 liters, 80 liters, 100 liters, etc.In the present example, a smartphone can be used as the client, and thesmartphone can include a high-definition camera and a screen. Asmartphone holder can be disposed near a user handrail or a holding partof the shopping cart, and can be used to place smartphones of aplurality of sizes and types. The smartphone can be placed in theholder, so that the camera of the smartphone faces towards the shoppingcart and can photograph the shopping cart. The screen of the smartphonefaces towards the user, and the screen can display generated user orderinformation (commodities placed in the shopping cart by the user) ordisplay a shopping prompt, an error, navigation information, and otherinformation.

It can be considered that the user places commodities into the shoppingcart one by one during shopping. When the user determines to use acertain shopping cart, it can be considered that a client (a smartphone)on the shopping cart is bound to the user. For example, if the userunlocks shopping cart A through scanning, shopping cart A is bound oneto one with an ID of the user after verification succeeds. In such acase, the smartphone on the shopping cart can run a photographingapplication. The smartphone can include an image change detectionmodule, for example, an image change detection algorithm or a computingmodel that is set in the photographing application, and can capturedifferent images or record a video containing the different imagesbefore and after an image change moment. In an implementation method,the smartphone can photograph the shopping cart at a predeterminedfrequency or keep photographing the shopping cart after enabling thephotographing application. In addition, the smartphone can detect acaptured image, and can determine, by using an image recognition method,whether a change of the goods occurs in the shopping cart, for example,the user puts in a new goods or takes out a goods. The image recognitionmethod can include obtaining a direct different part of the image anddetermining an image change by identifying the different part. Inspecific implementation, if an image difference absolute value in aspecified area exceeds a specific threshold, updated image frames can besent to the server at a specific time interval based on anadjacent-frame subtraction method. If an image remains unchanged, it canbe considered that no change occurs to the goods and no image needs tobe sent.

Therefore, in an implementation of the method provided in the presentspecification, the detecting, based on an image recognition method,whether a change of the goods occurs in a storage container can includethe following.

S02: Capture goods images of the storage container by using aphotographing device.

S04: Detect difference data between the goods images based on achronological sequence.

S06: If the difference data is greater than a predetermined changethreshold, determine that a change of the goods occurs in the storagecontainer.

The photographing device can include the previously described camera,and can also include a digital camera, a video camera, a monitor, etc.After obtaining the goods images of the storage container, the goodsimages can be detected based on a chronological sequence of frames, todetermine whether a change of the goods occurs in the goods storagecontainer. Generally, when a new goods is placed into or a goods istaken out from the storage container such as a shopping cart, an imageof the storage container captured by the photographing device changes.FIG. 3 is a schematic diagram illustrating an implementation scenario ofphotographing an added goods in a shopping cart by a client, accordingto one implementation of the present specification. The client candetect the image change, and can obtain the difference data for theimage change. In the present implementation, if the difference data forthe image change is greater than the predetermined change threshold, itcan be determined that a change of the goods occurs in the storagecontainer. As such, image change interference information not caused byputting in or taking out some commodities can be alleviated, forexample, a small shift of a goods in the shopping cart caused by shakingof the shopping cart. Certainly, a specific change threshold value rangefor determining the difference data can be set based on experience orexperiments. In the schematic diagrams respectively illustrating anempty shopping cart and a shopping cart with one goods in FIG. 2 andFIG. 3, for ease of image detection, shelters can be placed around or atthe bottom of the shopping cart. As such, when a goods is put in an areacaptured by the photographing device, the captured goods image changesobviously, and interference information outside the captured shoppingcart such as aisles, shelves, and silhouettes can be reduced.

Notably, the client photographs a goods in the storage container,including image data captured at a predetermined frequency, or includinggoods image data of a recorded video. In one or more implementations ofthe present specification, the video can be considered as a set ofcontinuous images or one type of image data. Therefore, the image datasuch as the goods image and an associated image in the presentspecification can include a separately captured image, and can alsoinclude an image in a video.

S2: If a change of the goods is detected, the client sends an associatedimage of the change of the goods in the storage container to a server.

If the client detects a change of the goods in the storage container,for example, a new goods is put in the storage container, the client candetect that the captured image has changed. When determining, based onthe captured goods image, that a change of the goods occurs in theshopping cart, the client can obtain the associated image of the changeof the goods in the shopping cart, and can send the associated image toa predetermined server for processing, for example, a cloud server usedfor supermarket order processing.

The associated image can include the image that is sent by the client tothe server for detecting and identifying a change of the goods, and canbe a single image or a plurality of images. In an implementation method,the associated image can include a separate discontinuous image capturedby the client side, or can include video data that is recorded by theclient and detects a change of the goods. In such a case, the video datacan be considered as a continuous associated image. During associatedimage determining, once the client determines that a change of the goodsoccurs in the storage container, different images or a video detectedbefore and after the image change moment can be used as the associatedimage. Certainly, in other implementation methods, images in a period oftime (for example, 3 seconds) before and after the image change momentcan also be used as the associated image. In one or more implementationsprovided in the present specification, the associated image can includeat least one of the following: S20: a goods image corresponding todifference data that is greater than the change threshold; S22: a goodsimage in a goods image predetermined window range corresponding to achange of the goods moment if it is determined that a change of thegoods occurs in the storage container; and S24: a goods image in a goodsimage predetermined time range corresponding to a change of the goodsmoment if it is determined that a change of the goods occurs in thestorage container.

In a specific application scenario, for example, when it is detectedthat difference data data0 between goods images at moment T0 and momentT1 is greater than the predetermined change threshold, goods images P0and P1 at moment T0 and moment T1 corresponding to the difference datadata0 can be used as the currently detected associated images for thechange of the goods in the storage container. Or, when a change of thegoods is detected at moment T2, goods image P2 corresponding to momentT2 and 10 goods images before and after P2 (in such a case, a windowsize is 11 images) are selected as the associated images to be sent tothe server. Or, image data recorded in a video shot in 3 seconds beforeand after image change moment T3 is selected as the associated image. Inthe implementations provided in the present specification, a method fordetermining the associated image can include one or a combination of thepreviously described methods. For example, the goods image correspondingto the difference data and 10 images (an overlapped image does not needto be selected) before and after a change moment can be jointly selectedas the associated images.

In the application scenario in the present implementation, duringprocessing of detecting whether a goods image is changed, whether achange of the goods occurs in the shopping cart can be determined byusing difference data obtained through image subtraction. FIG. 4 is aschematic diagram illustrating identification of goods boundaries for achange of the goods, according to one implementation of the presentspecification. L is identified boundary frames of a pizza packing box.Image boundaries of a goods newly added to the shopping cart as shown byL in FIG. 4 can be identified through image subtraction and connectedcomponent analysis with thresholding. In another implementation of themethod provided in the present specification, the client can detectgoods boundaries in an image that has a change of the goods, and thenupload an associated image to the server for goods identification, orderprocessing, etc. for the image. The client can upload an image withinthe identified goods boundaries to reduce image data transmission,network traffic, and server load. Therefore, in another implementationof the method in the present application, the method can further includethe following step:

S3: The client identifies goods boundaries for a change of the goods inthe associated image after obtaining the associated image.

Correspondingly, the client sends the associated image to the server,including sending an image within the identified goods boundaries to theserver.

In a specific example, after determining the goods boundaries shown inFIG. 4, a goods image shown in FIG. 5 that is within the goodsboundaries in FIG. 4 is uploaded to the server. FIG. 5 shows an imagewithin identified goods boundaries in an implementation scenario,according to one implementation of the present specification. It can beseen from FIG. 4 and FIG. 5 that, by uploading image information withinthe goods boundaries identified by the client in the presentimplementation, network traffic can be effective reduced, retransmittedredundant information can be reduced, image data processing load of theserver can be reduced, and image recognition processing efficiency canbe improved, etc.

Certainly, other goods boundary identification algorithms canalternatively be used, including the following case: A goods image for achange of the goods can still be identified when commodities arestacked. For example, FIG. 6 is a schematic diagram illustrating ascenario that another goods is stacked on the first goods in theshopping cart shown in FIG. 3, and FIG. 7 shows an image within goodsboundaries, of the new goods in FIG. 6, identified based on an imagedifference.

The client sends the associated image to the server. The client can sendthe obtained associated image to the server through near fieldcommunication such as WiFi or a local area network of a mobile phone ornetwork communication connections such as an operator's communicationsnetwork and a dedicated cable. The server can store the associated imageuploaded by the client. In an implementation method, the client candirectly send the associated image to the server, for example, throughan operator's communications network. In other implementation methods, auniversal interface device for a plurality of clients can be set, sothat a plurality of clients in a supermarket can send, through nearfield communication, the associated image to the universal interfacedevice, for example, a supermarket hosting server, and then thesupermarket hosting server can send the associated image to the serverused for identifying a goods in the image.

S4: The server identifies information about the goods difference in thestorage container based on the associated image.

After obtaining the associated image uploaded by the client, the servercan analyze and process the associated image, for example, performassociated image comparison based on a chronological sequence todetermine a difference, and identify information about a goods that isadded or removed in the associated image. An image processed by theserver can include an image captured by the client or data of avideo-type goods image, or can include image data within goodsboundaries identified by the client based on the image difference.

After analyzing and processing the associated image, the server canobtain the goods difference information for the change of the goods inthe storage container. The goods difference information can includeimage data, and can also include identification information that shows achange occurs in the goods in the storage container. For example, asshown in FIG. 3 and FIG. 4, it can be identified that a new goods is putin the shopping cart, and it can be determined through image recognitionthat the goods is G01, with a goods identification code of good_001. Insuch a case, it can be determined that the information about the goodsdifference in the storage container can be ADD:good_001, where ADD canindicate the following: The present change of the goods is that a goodsis added. Therefore, in another implementation of the method provided inthe present specification, the server identifies the information aboutthe goods difference in the storage container based on the associatedimage, including:

S40: Detect a goods identifier in the associated image, to use thedetected goods identifier as the goods difference information.

In a specific example, for example, the server can be set with a barcodedetection and identification module to identify a barcode in theassociated image, where the barcode can include a plurality of goodsattributes, for example, a goods name and a goods price. When a changeof the goods occurs in the shopping cart, the barcode detection andidentification module detects and identifies an input associated image,and searches for a regular barcode. If the identification succeeds, thegoods difference information can be determined.

Certainly, other implementation methods for detecting and identifying agoods in an associated image can further be set on the server side. Inanother implementation of the method provided in the presentspecification, a machine learning algorithm can be used to detect agoods in an associated image. Any of the following implementationmethods can be used:

S42: After obtaining the associated image, detect goods information inthe associated image by using a goods detection model obtained through amachine learning algorithm, to determine the goods differenceinformation.

S44: Detect a goods identifier in the associated image, to use thedetected goods identifier as the goods difference information.

When the goods identifier in the associated image fails to beidentified, goods information in the associated image is detected byusing a goods detection model obtained through a machine learningalgorithm, to determine the goods difference information.

Support Vector Machine (SVM), Regions with Convolutional Neural Networks(RCNN, a target detection algorithm), and other machine learningalgorithms and variants can be selected as the goods detection model.The selected machine learning algorithm can use sample data to performtraining and construction in advance. When an actual scenario or adesign-needed condition is satisfied, the algorithm for detecting agoods in an associated image in the present implementation can be used.In an implementation method, after the associated image is obtained, theassociated image can be input to the goods detection model for goodsdetection. In another implementation method, a goods identifier in anassociated image can be identified first, for example, a goods barcodeor QR code, etc. can be detected. When the goods identifier fails to beidentified, a higher-level machine learning algorithm can be used todetect a goods in the associated image, to determine goods information.

S6: The server generates corresponding goods order change information byusing the difference information.

After determining the difference information for the change of the goodsin the storage container, the corresponding order change information canbe generated based on the difference information. For example, when acorresponding goods is detected and a corresponding only price isidentified, goods order change information can be generated. The goodsorder information can indicate that the user adds one goods, and can beused for total order settlement processing of the user.

S8: The server updates user order information corresponding to theclient based on the goods order change information.

The server side can record total order information of the user. In animplementation, when the user unlocks the client and uses the storagecontainer for goods selection and purchase, the client can be bound toan identifier of the user, for example, one user is bound to oneshopping cart. When identifying a goods that is newly put into or takenout from the shopping cart by the user, the server can generate thecorresponding goods order change information, and then can accordinglyupdate, based on the goods order change information, the user orderinformation recorded on the server side, for example, a shopping listwith an item added or a shopping list with an item removed.

S10: The server sends updated user order information to the client.

In the implementation scenario in the present implementation, the clientside can display order information of an identified goods in theshopping cart to the user, and the user can obtain the order informationimmediately. When the user order information recorded on the server sideis updated, the server side can send updated user order information tothe client, so that the client can display the user order information tothe user or synchronize the user order information.

S12: The client displays the updated user order information.

When the user puts in a new goods or takes out a goods, the client canreceive the user order information updated after the server identifiesthe goods that is newly puts in or taken out by the user, and then candisplay the updated user order information, for example, display theuser order information on the smartphone installed on the shopping cart,or display the user order information on another display device disposedon the storage container.

According to the goods order processing method provided in one or moreimplementations of the previous implementations, a photographing devicecan be installed in a shopping basket, a shopping cart, or other storagecontainers, and a remote server can identify and determine a user orderchange by monitoring a change of the goods in the storage container inreal time, and can generate or update a shopping list of a user. Assuch, end-user devices such as an intelligent shopping cart can be usedto process data of self-service supermarket orders in combination with aserver for goods identification and order processing to reduceself-service cashier devices, sensor devices, etc. With theimplementation solutions provided in the present specification, an imagecapture device or a video shooting device of a client can be used tosend image data of goods detection to a server, so that the serverperforms processes such as goods identification and order settlement.Therefore, overall hardware implementation costs can be lower. Inaddition, data exchanging nodes can be reduced and solutionimplementations can be simpler and more convenient. In actualapplications, implementation costs of the technical solutions forautomatic settlement processing of self-service supermarket goods orderscan be greatly reduced, and the technical solutions are convenient toimplement and have better stability. An implementation environment andimplementation factors such as self-service supermarket size, customerflow volume, and consumer behavior feature have relatively small impact,and facilitate large-scale promotion and use.

Further, the present specification provides another implementation ofthe method. During identification and detection on the associated imageuploaded by the client, if the server side fails to identify a goods inthe associated image, for example, fails to scan a barcode or even failsto identify the goods in the associated image by using a goods detectionmodel obtained through machine learning, the server side can sendinformation about the associated image and/or user information andstorage container information to an end-user device used for manualidentification, for example, a PC end at a manual cashier desk, adedicated cashier device, or another end-user device disposed forprocessing a goods detection and identification failure. In the presentimplementation, the previous end-user devices including the end-userdevice used for manually identifying and processing an image recognitionfailure can be collectively referred to cashier nodes. As such, when theserver fails to identify the goods in the image, the server can send theimage to the cashier node for manual identification processing, whichcan effectively ensure timely, stable, and continuous processing ofgoods orders and reducing user shopping experience deterioration causedby goods identification failures on the server side.

In the present implementation, one or more cashier nodes can bedisposed. Generally, a plurality of cashier nodes can be disposed in asupermarket. Each cashier node can correspond to one network cashier,and the network cashier can operate a device on the cashier node,identify or discard a received goods image, remind a customer, etc.Therefore, in some implementations of the present application, a taskscheduling queue and a network cashier category can be set, to performmore proper task scheduling on a plurality of unidentifiable goodsimages in a unified manner. In another implementation of the methodprovided in the present specification, the method can further includethe following steps:

S46: If a goods identifier in the associated image fails to be detected,or fails to be detected by using the goods detection model, generate anoperation task for the detection failure of the associated image, andplace the operation task in a task scheduling queue.

S48: Push, based on an obtained network cashier list, the operation taskin the task scheduling queue to cashier nodes that satisfy operationcondition, where the network cashier list records processing taskoperation statuses of cashier nodes.

The operation task can include to-be-processed data information formedbased on received information about the image that fails to beidentified and other information such as a user identifier and a clientidentifier. The operation task in the task scheduling queue can beallocated to a cashier node, and the cashier node can performcorresponding processing based on task information. The network cashierlist can further be configured. The network cashier list can recordcurrent operation statuses of cashier nodes, for example, whether acashier node is online and how many unprocessed tasks that a certaincashier node currently has, etc. Based on the operation statuses of thecashier nodes in the network cashier list, the tasks in the taskscheduling queue can be allocated to the cashier nodes that satisfyoperation condition. The operation condition can be specified based onan on-site implementation scenario or a design need. For example, thefollowing can be specified: When a certain channel is used for manualidentification, one or more or all operation tasks can be allocated to aspecified cashier node for processing, or the operation tasks can beallocated to a cashier node currently with less small operation amountfor processing.

Notably, in other implementations, the cashier node is not limited to anend-user for manual identification processing, and can also include anend-user device with a storage, recording, forwarding, or alarmingfunction.

The present specification further provides another implementationmethod. In some implementation scenarios, when the user fails to selecta shopping cart or the server side fails to identify a goods, anidentifier of the user is bound to a network cashier. Or, when nonetwork cashier is processing an operation task of a user, a networkcashier can be bound to the operation task of the user in the taskscheduling queue. As such, when an operation task in a task schedulingqueue is allocated, it is first queried whether a network cashier isbound to the task, and if yes, the operation task is allocated to acashier node of the bound cashier with priority. The bound cashier caninclude a cashier that is processing an operation task that belongs tothe same user as the operation task to be allocated, or a certaincashier specified by the server or the user. Certainly, the binding canalso include an implementation method of binding the client to thenetwork cashier. In another implementation of the method provided in thepresent application, the method can further include the following step:

S47: Query whether a network cashier in the network cashier list isbound to the operation task.

If yes, the corresponding operation task is sent to a cashier node ofthe corresponding bound network cashier for processing.

In another implementation method, if an operation amount of the boundnetwork cashier has reached a saturated state, for example, a specifiedmaximum processing amount, the operation task can be sent to anothernetwork cashier for processing, to balance operation processing load andimprove processing efficiency. Therefore, in another implementation ofthe method in the present application, the method can further includethe following step:

S471: After determining the network cashier bound to the operation task,query an operation status of the bound cashier node.

If the operation status indicates a saturated operation amount, theoperation task is sent to a cashier node of a network cashier with anunsaturated operation amount in the network cashier list for processing.

In a specific example, for example, a task queue can be saved and eachitem in the task queue includes a user ID+an associated image that failsto be identified or video content+an order corresponding to a user ID.In addition, a network cashier list and a task list of each networkcashier can be saved. When there is a new operation task for anassociated image recognition failure, a user ID is searched to determinewhether a network cashier has been bound for processing. If yes, it isdetermined whether an operation mount of the network cashier isunsaturated. If not, the operation task can be pushed to the networkcashier with priority. If the operation mount of the network cashier issaturated, the operation task can be pushed to a new network cashier forprocessing, and the user ID is bound to an ID of the new cashier. If theoperation task corresponding to the user ID is currently not processedby a network cashier, the operation task can be pushed to a new networkcashier for processing, and the user ID is bound to an ID of the newnetwork cashier. The new network cashier can be a specified cashier withan unsaturated operation amount or a randomly selected cashier or anetwork cashier that is selected based on a predetermined sequence.

In another implementation, the operation task (generally including anassociated image of a goods identification failure) can be alternativelypushed to a plurality of cashiers for processing, and the plurality ofcashiers can mutually check, reference, or combine their results, etc.In another implementation of the method in the present specification,the pushing, based on an obtained network cashier list, the operationtask in the task scheduling queue to cashier nodes that satisfyoperation condition includes the following: pushing the operation taskto at least two cashier nodes; and correspondingly, the identifyinginformation about the goods difference in the storage container based onthe associated image includes: determining the information about thegoods difference in the storage container based on goods identificationresults of the at least two cashier nodes.

For example, in some application scenarios with relatively high securityneeds, an operation task corresponding to one user ID can be pushed to aplurality of cashiers for processing, and the plurality of cashiers canmutually check and review their results. As such, goods identificationaccuracy can be further improved.

Further, if a network cashier on a cashier node cannot identify a goodsin a goods image captured and uploaded by the client either, the networkcashier can send a prompt message to the client. The prompt message caninclude a plurality of types of information, for example, to notify theuser that no order can be generated, or return a result of a goodsidentification failure to the user, to prompt the user about anidentification failure. Or, the prompt message can include informationused for prompting the user that no order can be generated for a goodsthat is just put in and the goods need to be re-arranged as needed, orinformation used for directly identifying an operation task as anunidentified-goods order. In another implementation of the method in thepresent application, the method can further include the following: if agoods identification result of the cashier node includes a goodsidentification failure, one of the following is performed:

S691: The server returns a prompt message to the client, where theprompt message includes at least one of information content that a goodsorder generation failure and a goods re-arrangement request.

S692: Generate an unidentified-goods order corresponding to anassociated image for the goods identification failure, and set, based oninformation about the unidentified-goods order, an identity verificationresult of a storage container corresponding to the unidentified-goodsorder to “failed”.

In S692, when the storage container needs to pass through a channelapparatus, opening and closing of a gate of the channel apparatus can becontrolled based on a result of querying whether there are unidentifiedgoods in the storage container.

If a goods fails to be identified, and the goods that fails to beidentified is still in the storage container, the gate of the channelapparatus can be controlled to not open or to close when the storagecontainer needs to pass through an exit. In such a case, the user canmove to a manual cashier channel for processing. The channel can includean apparatus that is used in supermarkets, shopping malls, or metrostations for controlling a channel gate to open or close for people orstorage containers to pass through. In a specific example, the user IDcan be bound to an ID of a used shopping cart. When the user pushes theshopping cart to approach a self-service settlement channel, the user IDor the ID of the shopping cart can be identified through identificationcode scanning infrared, or other near field communication forself-service settlement. Then, it can be queried at a back end whetherthere are unidentified goods in a record corresponding to the user ID orthe ID of the shopping cart. If there is an unidentified goods in theshopping cart, the shopping cart cannot pass through the self-servicesettlement channel (which can be controlled by controlling, based onidentity verification, the channel gate to open or close), and needs tomove to the manual cashier channel for further processing.

The one or more goods order processing methods provided in the previousimplementations in the present specification can provide an ordersettlement method that facilitates users' shopping and has lowerimplementation costs for merchants. In self-service supermarketapplications, a user's shopping list can be identified and confirmed bymonitoring a change of the goods in real time (processed through imagerecognition), and a remote server can generate the user's total order inreal time. Order settlement is automatically performed after the usercompletes shopping, and the user can directly leave the self-servicesupermarket after completing payment. In such a case, data exchangingnodes are less, there is no need to configure a large number ofexpensive cashier devices, light curtains, and other sensor devices,implementation costs are lower, and order data processing achieveshigher efficiency and is more stable.

Further, the present specification provides another implementationmethod. The client or the storage container corresponding to the clientcan use a positioning apparatus, for example, a GPS positioning methodor a Ultra Wideband (UWB, which is a carrierless communicationstechnology that uses a non-sinusoidal narrow impulse from a nanosecondlevel to a picoseconds level to transmit data, and can be used foraccurate positioning, particularly, indoor positioning currently)positioning method. An identifier of a certain goods distribution rangecan be set in the area, commodities in the area can be placed to thesame interval, and each interval can correspond to goods information inthe interval. As such, when commodities are distributed to differentintervals, the goods selected by the user are usually the goodsdistributed within the intervals. Therefore, a goods interval of a goodsselected by the user can be determined in combination with positioningof the storage container and space interval distribution of the goods.When goods cannot be directly identified by using an associated image,the goods can be identified in a goods information range of the goodsinterval. As such, goods search intervals can be greatly reduced, imagerecognition search intervals can be reduced, identification speed andidentification success rate can be increased, and identificationprocessing load can be reduced. Therefore, in another implementation ofthe method in the present application, the method can further includethe following step:

S14: The client obtains position information of the storage container,and sends the positioning information to the server.

The server determines a goods interval for the change of the goods inthe storage container based on the positioning information.

Correspondingly, the server identifies the information about the goodsdifference in the storage container based on the associated image,including: identifying goods information in the associated image in arange of the goods interval, and determining the difference informationfor the change of the goods in the storage container based on theidentified goods information.

FIG. 8 is a schematic flowchart illustrating another implementation ofthe method, according to one implementation of the presentspecification. Specific implementations of the present specification aredescribed above. Other implementations fall within the scope of theappended claims. In some situations, the actions or steps described inthe claims can be performed in a sequence different from the sequence inthe implementation and the desired results can still be achieved. Inaddition, the process depicted in the accompanying drawings does notnecessarily need a particular execution sequence to achieve the desiredresults. In some implementations, multi-tasking and parallel processingcan be advantageous.

The server in the previous implementations can include a separateserver, or can be divided into different logical processing units insome application scenarios. For example, the server can include aplurality of logical end-user devices such as a module configured toreceive and store an image uploaded by the client, a module configuredto analyze the image and identify a goods, a module configured toperform manual task scheduling when automatic goods identificationfails, and a network cashier management module. These logical end-userdevices can be collectively considered as the server side, or can beseparate devices connected or coupled to the server. Similarly, theclient can include a photographing device and a communication devicethat are installed on the shopping cart. In other application scenarios,the photographing device, the communication device, and the shoppingcart can be jointly considered as the client, or can be considered asthe client together with the positioning device on the shopping cart. Inone or more implementations of the present specification, the client andthe server can be divided based on different data processingmethods/different data processing phases of image detection apparatus 1used for detecting whether a change of the goods occurs in the storagecontainer and apparatus 2 for image recognition and order processing.The client, the server, and various function modules on the server sidecan be described based on an actual application environment.

FIG. 9 below is a schematic diagram illustrating an implementationscenario of a goods order processing method, according to oneimplementation of the present specification. In FIG. 9, a holder can beinstalled on a shopping cart, to place a mobile phone or a networkcamera. A server side can be divided into different processing units.Details are as follows:

101: The shopping cart can be a common shopping cart in a shopping mallor a supermarket, and can have capacities such as 60 liters, 80 liters,100 liters, 125 liters, etc. The holder can be an external mobile phoneholder, and can be fastened in front of a handrail of the shopping cart.It can be considered that a user places commodities into the shoppingcart one by one during shopping.

100: The mobile phone can include a high-definition rear camera and ascreen, and can be placed in the holder in 101. An angle can be adjustedwhen the holder and the mobile phone are installed, so that the screenis tilted towards the user, and the camera can photograph the shoppingcart. The mobile phone runs a photographing application, and can includean image change detection module, to record different images or a videobefore and after an image change moment for uploading to a cloud server.The changed image areas are uploaded to the cloud, so that an automaticor manual analysis unit on the server side can process the images. Here,one mobile phone can be bound to one user. When the user confirms to usea certain shopping cart through scanning or in other ways, a mobilephone on the shopping cart can be bound to an identifier (ID) of theuser.

102: The cloud server or a back end service is connected to andcommunicates with the mobile phone through WiFi or an operator'snetwork, and can store an image and a video of the shopping cartcaptured by the camera. The cloud server can be responsible for storinguser orders, and inputting content to be analyzed to 105 and 103.

105: An automatic video analysis unit can include a barcode detectionand identification module and an alarming module. In otherimplementations, a machine-learning-based goods detection module can beincluded. Machine-learning-based detection can be performed based onmethods such as SVM and RCNN. When a video and an image are changed, thebarcode detection and identification module inputs different images, andsearches for a regular barcode. If the identification succeeds, goodsare added to the user's shopping list. If the barcode detection fails,the image and video information is sent to the machine-learning-basedgoods detection module. If the machine-learning-based goods detectionmodule detects corresponding goods and identifies a corresponding onlyprice, the goods can be added to the user's shopping list. If thedetection fails, the image can be sent to the alarming module. Thealarming module can encapsulate the information into an operation taskand send the operation task to a manual task scheduling unit. Theoperation task can include information such as a user ID+an alarmingimage or video content+a current order corresponding to the user ID.

103: The manual task scheduling unit can store a task queue, and eachitem in the task queue can include the user ID+the alarming image or thevideo content+the order corresponding to the user ID. In addition, themanual task scheduling unit can store a network cashier list and a tasklist of each network cashier. When there is a new alarming task item, auser ID can be searched, to determine whether a network cashier is boundfor processing. If yes, it is determined whether an operation amount ofthe network cashier is saturated. If the operation amount isunsaturated, the operation task is pushed to the cashier with priority.If the operation mount of the network cashier is saturated, theoperation task can be pushed to a new cashier for processing, and theuser ID is bound to an ID of the new cashier. If the operation taskcorresponding to the user ID is currently not processed by a networkcashier, the operation task can be pushed to new network cashier forprocessing, and the user ID is bound to an ID of the new networkcashier. For example, as described in some previous implementations, insome application scenarios with relatively high security needs, anoperation task corresponding to one user ID can be pushed to a pluralityof cashiers for processing, and the plurality of cashiers can mutuallycheck and review their results.

104: A network cashier manual operation and control unit can beconnected to and communicate with the manual task scheduling unitthrough an Ethernet or an operator's network. A screen can be set, todisplay the alarming image or video. The network cashier can be a realoperation person, and the network cashier can observe the alarming imagewith eyes, to search for and identify a barcode area, and add goods to auser order after the identification succeeds. If the barcodeidentification fails (for example, due to blocking or other reasons),the network cashier can search the store's goods list for acorresponding goods, and add the goods to the user order upon a searchsuccess. If the search fails, the network cashier can mark the goods andreturn a result to 103. 103 can feed back the result to the user'sscreen 100, and remind the user that no order can be generated for thegoods that are just put in and the goods needs to be re-arranged, ordirectly put the goods in unidentified items in the user order.

In addition, in other implementation methods, the shopping cart in 101can be configured with a positioning method similar to UWB, to providenavigation help for customers and collect statistics on a customer flow.Moreover, a goods Stock Keeping Unit (SKU, which can be piece, box,tray, etc.; the SKU is generally construed as a short name for a unifiedproduct number, and each product uniquely corresponds to one SKU number)range around a shelf can be learned through positioning, to reduce imagerecognition search space and improve a machine/manual goodsidentification success rate.

It can be seen from the descriptions in the previous implementationsthat, according to the goods order processing method provided in one ormore implementations of the present specification, a photographingdevice can be installed in a shopping basket, a shopping cart, or otherstorage containers, and a remote server can identify and determine auser order change by monitoring a change of the goods in the storagecontainer in real time, and can generate or update a shopping list of auser. As such, end-user devices such as an intelligent shopping cart canbe used to process data of self-service supermarket orders incombination with a server for goods identification and order processingto reduce self-service cashier devices, sensor devices, etc. With theimplementation solutions provided in the present specification, an imagecapture device or a video shooting device of a client can be used tosend image data of goods detection to a server, so that the serverperforms processes such as goods identification and order settlement.Therefore, overall hardware implementation costs can be lower. Inaddition, data exchanging nodes can be reduced and solutionimplementations can be simpler and more convenient. In actualapplications, implementation costs of the technical solutions forautomatic settlement processing of self-service supermarket goods orderscan be greatly reduced, and the technical solutions are convenient toimplement and have better stability. An implementation environment andimplementation factors such as self-service supermarket size, customerflow volume, and consumer behavior feature have relatively small impact,and facilitate large-scale promotion and use.

The previous implementations describe the implementation methods of oneor more goods order processing methods provided in the presentspecification from a perspective of multi-party interaction between theclient and the server, or in other words, between the end-user device onthe client side and the end-user device on the server side. Based on thedescriptions in the previous implementations, the present specificationfurther provides a goods order processing method on the server side. Theserver can include a separate server operation system, and canalternatively include a single server or a server cluster or adistributed system that has a plurality of processing units used forimage storage, image analysis, operation scheduling, etc. The presentspecification further provides the goods order processing method onavailable on the server side. FIG. 10 is a schematic flowchartillustrating an implementation of the method available on the serverside, according to one implementation of the present specification. Asshown in FIG. 10, the method can include the following steps:

S100: Receive an associated image uploaded by a client, where theassociated image includes an image obtained when the client detects,based on an image recognition method, that a change of the goods occursin a storage container.

S101: Identify information about the goods difference in the storagecontainer based on the associated image.

S102: Generate corresponding goods order change information by using thedifference information.

S103: Update user order information corresponding to the client based onthe goods order change information.

S104: Send updated user order information to the client.

In an implementation of the method, the associated image received by theserver can include an image or a video image (a video is considered ascontinuous images here) obtained when the client detects that a changeof the goods occurs in the storage container. The client can detect andidentify an image obtained by photographing the storage container, todetect and determine whether a change of the goods occurs in the storagecontainer. For example, subtraction of adjacent goods images can beused, so that when an image difference absolute value in a specifiedarea exceeds a specific threshold, it can be considered that a change ofthe goods occurs in the storage container. The detecting, based on animage recognition method, whether a change of the goods occurs in astorage container can include the following:

S1001: Obtain goods images of the storage container captured by using aclient photographing device.

S1002: Detect difference data between the goods images based on achronological sequence.

S1003: If the difference data is greater than a predetermined changethreshold, determine that a change of the goods occurs in the storagecontainer.

The associated image can include at least one of the following images:S1004: a goods image corresponding to difference data that is greaterthan the change threshold; S1005: a goods image in a goods imagepredetermined window range corresponding to a change of the goods momentif it is determined that a change of the goods occurs in the storagecontainer; and S1006: a goods image in a goods image predetermined timerange corresponding to a change of the goods moment if it is determinedthat a change of the goods occurs in the storage container.

The identifying information about the goods difference in the storagecontainer based on the associated image can include the following:

S1011: Detect a goods identifier in the associated image, to use thedetected goods identifier as the goods difference information.

In another implementation of the method, any of the followingimplementation methods can be used:

S1012: After obtaining the associated image, detect goods information inthe associated image by using a goods detection model obtained through amachine learning algorithm, to determine the goods differenceinformation.

S1013: Detect a goods identifier in the associated image, to use thedetected goods identifier as the goods difference information.

When the goods identifier in the associated image fails to beidentified, goods information in the associated image is detected byusing a goods detection model obtained through a machine learningalgorithm, to determine the goods difference information.

In another implementation of the method provided in the presentspecification, the method can further include the following steps:

S1014: If goods in the associated image fails to be identified, generatean operation task for the detection failure of the associated image, andplace the operation task in a task scheduling queue.

S1015: Push, based on an obtained network cashier list, the operationtask in the task scheduling queue to cashier nodes that satisfy anoperation condition, where the network cashier list records processingtask operation statuses of cashier nodes.

In another implementation of the method provided in the presentspecification, the method can further include the following steps:

S1016: Query whether a network cashier in the network cashier list isbound to the operation task.

If yes, the corresponding operation task is sent to a cashier node ofthe corresponding bound network cashier for processing.

In another implementation of the method provided in the presentspecification, the method can further include the following steps:

S1017: After determining the network cashier bound to the operationtask, query operation status of the bound cashier node.

If the operation status indicates a saturated operation amount, theoperation task is sent to a cashier node of a network cashier with anunsaturated operation amount in the network cashier list for processing.

Further, in another implementation of the method in the presentspecification, the pushing, based on an obtained network cashier list,the operation task in the task scheduling queue to cashier nodes thatsatisfy an operation condition includes the following step:

S1018: Push the operation task to at least two cashier nodes.

Correspondingly, the identifying information about the goods differencein the storage container based on the associated image includesdetermining the information about the goods difference in the storagecontainer based on goods identification results of the at least twocashier nodes.

In another implementation of the method provided in the presentspecification, the method can further include the following steps:

S1019: If a goods identification result of the cashier node includes agoods identification failure, perform one of the following: returning aprompt message to the client, where the prompt message includesinformation content of at least one of a goods order generation failureand a goods re-arrangement; and generating an unidentified-goods ordercorresponding to an associated image for the goods identificationfailure, and setting, based on information about the unidentified-goodsorder, an identity verification result of a storage containercorresponding to the unidentified-goods order to “failed”.

In another implementation of the method provided in the presentspecification, the method can further include the following steps:

S10110: Receive positioning information of the storage containeruploaded by the client, and determine a goods interval for the change ofthe goods in the storage container based on the positioning information.

Correspondingly, the identifying information about the goods differencein the storage container based on the associated image includesidentifying goods information in the associated image in a range of thegoods interval, and determining the difference information for thechange of the goods in the storage container based on the identifiedgoods information.

The previous implementations describe a plurality of implementations ofthe goods order processing method on the server side. For a detailedimplementation process and implementation method of the goods orderprocessing method, reference can be made to related descriptions on theimplementations of interaction between the client and the server, anddetails are omitted here.

Certainly, based on the description on the implementations ofinteraction between the client and the server, the present specificationfurther provides a goods order processing method available on the clientside. The client can also include an image change detection module, andcan detect a change of the goods in the storage container based on animage obtained by the photographing apparatus. In some otherimplementation scenarios, one or more of the shopping cart, the holderdevice, the screen, or the positioning device can be considered as theclient together with the photographing apparatus and a datacommunication apparatus used for image transmission. The method can beapplied to the client side, and can detect a change of the goods duringshopping, and send an image or a video obtained when the change of thegoods occurs to the server for identification and processing, togenerate the user's order information. The present specificationprovides a goods order processing method available on the client side.FIG. 11 is a schematic flowchart illustrating an implementation of themethod available on the client side, according to one implementation ofthe present specification. As shown in FIG. 11, the method can includethe following steps.

S200: Detect, based on an image recognition method, whether a change ofthe goods occurs in a storage container.

S201: If a change of the goods is detected, send an associated image ofthe change of the goods in the storage container to a server.

S202: Receive updated user order information returned by the server.

S203: Display the updated user order information.

Reference can be made to the previous implementations. In another goodsorder processing method available on the client side provided in thepresent specification, the detecting, based on an image recognitionmethod, whether a change of the goods occurs in a storage container caninclude the following steps.

S2001: Capture goods images of the storage container by using aphotographing device.

S2002: Detect difference data between the goods images based on achronological sequence.

S2003: If the difference data is greater than a predetermined changethreshold, determine that a change of the goods occurs in the storagecontainer.

The associated image can be determined in a plurality of ways. Inanother implementation of the method provided in the presentspecification, the associated image can include at least one of thefollowing images: S2004: a goods image corresponding to difference datathat is greater than the change threshold; S2005: a goods image in agoods image predetermined window range corresponding to a change of thegoods moment if it is determined that a change of the goods occurs inthe storage container; and S2006: a goods image in a goods imagepredetermined time range corresponding to a change of the goods momentif it is determined that a change of the goods occurs in the storagecontainer.

FIG. 12 is a schematic flowchart illustrating another implementation ofthe method available on the client side, according to one implementationof the present specification. In another implementation, when uploadingthe associated image, the client can send only the images that showchanges to goods to the server for processing, to save network bandwidthand increase processing speed. In another implementation of the method,after the associated image is obtained, the method can further includethe following step.

S2007: Identify goods boundaries for a change of the goods in anassociated image.

Correspondingly, the sending an associated image to a server caninclude: sending an image within the identified goods boundaries to theserver.

Further, a positioning method similar to UWB can be configured on theshopping cart, to provide navigation help for customers and collectstatistics on a customer flow. Moreover, a goods SKU range around ashelf can be positioned, to reduce image recognition search space andimprove machine/manual goods identification success rate. In anotherimplementation of the method, the method can further include thefollowing step:

S2008: Obtain positioning information of the storage container, andsending the positioning information to the server, so that the serverdetermines a goods interval for a goods in the storage container basedon the positioning information.

It can be seen from the plurality of implementations for interactionbetween the client and the server, the server side, and the client side,the goods order processing method provided in the present specificationcan be applied to application scenarios with consumer self-serviceshopping, platform-based automatic goods identification, and ordersettlement in self-service supermarkets. According to theimplementations provided in the present specification, a photographingdevice can be installed in a shopping basket, a shopping cart, or otherstorage containers, and a remote server can identify and determine auser order change by monitoring a change of the goods in the storagecontainer in real time, and can generate or update a shopping list of auser. As such, end-user devices such as an intelligent shopping cart canbe used to process data of self-service supermarket orders incombination with a server for goods identification and order processingto reduce self-service cashier devices, sensor devices, etc. With theimplementation solutions provided in the present specification, an imagecapture device or a video shooting device of a client can be used tosend image data of goods detection to a server, so that the serverperforms processes such as goods identification and order settlement.Therefore, overall hardware implementation costs can be lower. Inaddition, data exchanging nodes can be reduced and solutionimplementations can be simpler and more convenient. In actualapplications, implementation costs of the technical solutions forautomatic settlement processing of self-service supermarket goods orderscan be greatly reduced, and the technical solutions are convenient toimplement and have better stability. An implementation environment andimplementation factors such as self-service supermarket size, customerflow volume, and consumer behavior feature have relatively small impact,and facilitate large-scale promotion and use.

Based on the previous goods order processing method, one or moreimplementations of the present specification further provide a goodsorder processing apparatus. The apparatus can include an apparatus thatcombines necessary implementation hardware and uses a system (includinga distributed system), software (an application), a module, a component,a server, a client, etc. of the method in the implementations of thepresent specification. Based on a same innovative concept, the apparatusprovided in the one or more implementations of the present specificationis described in the following implementations. Because aproblem-resolving implementation solution of the apparatus is similar toa problem-resolving implementation solution of the method, for specificimplementation of the apparatus in the implementations of the presentspecification, reference can be made to the implementation of theprevious method. No repeated description is provided. A term “unit” or“module” used in the following implementations can implement acombination of software and/or hardware of a predetermined function.Although the apparatus described in the following implementations ispreferably implemented by software, implementation by hardware or acombination of software and hardware is possible to conceive. FIG. 13 isa schematic structural diagram illustrating modules of an implementationof a goods order processing apparatus available on a server side,according to one implementation of the present specification. As shownin FIG. 13, the apparatus can include the following: an image receivingmodule 101, configured to receive an associated image uploaded by aclient, where the associated image includes an image obtained when theclient detects, based on an image recognition method, that a change ofthe goods occurs in a storage container; an image recognition module102, configured to identify information about the goods difference inthe storage container based on the associated image; an order generationmodule 103, configured to generate corresponding goods order changeinformation by using the difference information; an order update module104, configured to update user order information corresponding to theclient based on the goods order change information; and an informationfeedback module 105, configured to send updated user order informationto the client.

In another implementation of the apparatus, the detecting, based on animage recognition method, whether a change of the goods occurs in astorage container can include the following: capturing goods images ofthe storage container by using a client photographing device; detectingdifference data between the goods images based on a chronologicalsequence; and if the difference data is greater than a predeterminedchange threshold, determining that a change of the goods occurs in thestorage container.

In another implementation of the apparatus, the associated image caninclude at least one of the following images: a goods imagecorresponding to difference data that is greater than the changethreshold; a goods image in a goods image predetermined window rangecorresponding to a change of the goods moment if it is determined that achange of the goods occurs in the storage container; and a goods imagein a goods image predetermined time range corresponding to a change ofthe goods moment if it is determined that a change of the goods occursin the storage container.

FIG. 14 is a schematic structural diagram illustrating modules of animplementation of an image recognition module in the apparatus,according to one implementation of the present specification. As shownin FIG. 14, the image recognition module 102 can include the following:an identification code detection module 1021, configured to detect agoods identifier in the associated image, to use the detected goodsidentifier as the goods difference information.

As shown in FIG. 14, the image recognition module 102 in animplementation of the apparatus can include the following: a goodsdetection module 1022, where the goods detection module 1022 includes agoods detection model obtained after sample training performed based ona selected machine learning algorithm, configured to detect goodsinformation in the associated image after the associated image isobtained, to determine the goods difference information; or configuredto detect goods information in the associated image when the goodsidentifier in the associated image fails to be identified, to determinethe goods difference information.

FIG. 15 is a schematic structural diagram illustrating modules ofanother implementation of the apparatus, according to one implementationof the present specification. As shown in FIG. 15, the apparatus canfurther include the following: a manual scheduling module 106,configured to, if a goods identifier in the associated image fails to bedetected, or fails to be detected by using the goods detection model,generate an operation task for the detection failure of the associatedimage, and place the operation task in a task scheduling queue; andpush, based on an obtained network cashier list, the operation task inthe task scheduling queue to cashier nodes that satisfy an operationcondition, where the network cashier list records processing taskoperation statuses of cashier nodes.

FIG. 16 is a schematic structural diagram illustrating modules ofanother implementation of the apparatus, according to one implementationof the present specification. As shown in FIG. 16, in anotherimplementation of the apparatus, the apparatus can further include thefollowing: a cashier control module 107, configured to query whether anetwork cashier in the network cashier list is bound to the operationtask; and if yes, send the corresponding operation task to a cashiernode of the corresponding bound network cashier for processing.

In another implementation of the apparatus, after determining thenetwork cashier bound to the operation task, the cashier control module107 can be further configured to query an operation status of the boundcashier node; and if the operation status indicates a saturatedoperation amount, send the operation task to a cashier node of a networkcashier with an unsaturated operation amount in the network cashier listfor processing.

In another implementation of the apparatus, the manual scheduling module106 pushes, based on the obtained network cashier list, the operationtask in the task scheduling queue to the cashier nodes that satisfy theoperation condition, including: pushing the operation task to at leasttwo cashier nodes; and correspondingly, the image recognition module 102identifies the information about the goods difference in the storagecontainer based on the associated image, including: determining theinformation about the goods difference in the storage container based ongoods identification results of the at least two cashier nodes.

In another implementation of the apparatus, a goods identificationresult of the cashier node includes a goods identification failure, theinformation feedback module 105 performs one of the following: returninga prompt message to the client, where the prompt message includesinformation content of at least one of a goods order generation failureand a goods re-arrangement; and generating an unidentified-goods ordercorresponding to an associated image for the goods identificationfailure, and setting, based on information about the unidentified-goodsorder, an identity verification result of a storage containercorresponding to the unidentified-goods order to “failed”.

FIG. 17 is a schematic structural diagram illustrating modules ofanother implementation of the apparatus, according to one implementationof the present specification. As shown in FIG. 17, in anotherimplementation of the apparatus, the apparatus can further include thefollowing: a positioning processing module 108, configured to receivepositioning information of the storage container uploaded by the client,and determining a goods interval for the change of the goods in thestorage container based on the positioning information; andcorrespondingly, the image recognition module 102 identifies theinformation about the goods difference in the storage container based onthe associated image, including: identifying goods information in theassociated image in a range of the goods interval, and determining thedifference information for the change of the goods in the storagecontainer based on the identified goods information.

Notably, the previous apparatus can further include anotherimplementation method based on the descriptions of the implementationsof interaction between the client and the server and the methodimplementations on the server side. For a specific implementationmethod, reference can be made to the descriptions in the related methodimplementations, and details are omitted here.

Specific implementations of the present specification are describedabove. Other implementations fall within the scope of the appendedclaims. In some situations, the actions or steps described in the claimscan be performed in a sequence different from the sequence in theimplementation and the desired results can still be achieved. Inaddition, the process depicted in the accompanying drawings does notnecessarily need a particular execution sequence to achieve the desiredresults. In some implementations, multi-tasking and parallel processingcan be advantageous. For example, the manual scheduling module 106 andthe cashier control module 107 can be combined into one processing unitor one function module to implement the steps of the modules.

The present specification further provides a goods order processingapparatus available on a client side. FIG. 18 is a schematic structuraldiagram illustrating modules of an implementation of a goods orderprocessing apparatus, according to one implementation of the presentspecification. As shown in FIG. 18, the apparatus can include thefollowing: a change of the goods detection module 201, configured todetect, based on an image recognition method, whether a change of thegoods occurs in a storage container; an image sending module 202,configured to, if a change of the goods is detected, send an associatedimage of the change of the goods in the storage container to a server;an information receiving module 203, configured to receive updated userorder information returned by the server; and a display module 204,configured to display the updated user order information.

FIG. 19 is a schematic structural diagram illustrating modules of animplementation of a change of the goods detection module in theapparatus, according to one implementation of the present specification.As shown in FIG. 19, the change of the goods detection module 201 caninclude the following: a photographing unit 2011, configured to capturegoods images of the storage container; an image difference processingunit 2012, configured to detect difference data between the goods imagesbased on a chronological sequence; and a change of the goods determiningmodule 2013, configured to, if the difference data is greater than apredetermined change threshold, determine that a change of the goodsoccurs in the storage container.

In another implementation of the apparatus, the associated image in theimage sending module 202 includes at least one of the following images:a goods image corresponding to difference data that is greater than thechange threshold; a goods image in a goods image predetermined windowrange corresponding to a change of the goods moment if it is determinedthat a change of the goods occurs in the storage container; and a goodsimage in a goods image predetermined time range corresponding to achange of the goods moment if it is determined that a change of thegoods occurs in the storage container.

FIG. 20 is a schematic structural diagram illustrating modules ofanother implementation of a change of the goods detection module in theapparatus, according to one implementation of the present specification.As shown in FIG. 20, the change of the goods detection module 201 caninclude the following: a change boundary identification module 2014,configured to identify goods boundaries for a change of the goods in theassociated image; and correspondingly, the image sending module 202sends the associated image to the server, including: sending an imagewithin the identified goods boundaries to the server.

FIG. 21 is a schematic structural diagram illustrating modules ofanother implementation of the apparatus, according to one implementationof the present specification. As shown in FIG. 21, the apparatus canfurther include the following: a location positioning module 205,configured to obtain positioning information of the storage container,and send the positioning information to the server, so that the serverdetermines a goods interval of a goods in the storage container based onthe positioning information.

Notably, the previous apparatus can further include anotherimplementation method based on the descriptions of the implementationsof interaction between the client and the server and the methodimplementations on the client side. For a specific implementationmethod, reference can be made to the descriptions in the related methodimplementations, and details are omitted here.

Specific implementations of the present specification are describedabove. Other implementations fall within the scope of the appendedclaims. In some situations, the actions or steps described in the claimscan be performed in a sequence different from the sequence in theimplementation and the desired results can still be achieved. Inaddition, the process depicted in the accompanying drawings does notnecessarily need a particular execution sequence to achieve the desiredresults. In some implementations, multi-tasking and parallel processingcan be advantageous. For example, the image sending module 202 and theinformation receiving module 203 can be combined into one communicationmodule to implement the steps of the modules.

According to the goods order processing apparatus provided in one ormore implementations of the present specification, a photographingdevice can be installed in a shopping basket, a shopping cart, or otherstorage containers, and a remote server can identify and determine auser order change by monitoring a change of the goods in the storagecontainer in real time, and can generate or update a shopping list of auser. As such, end-user devices such as an intelligent shopping cart canbe used to process data of self-service supermarket orders incombination with a server for goods identification and order processingto reduce self-service cashier devices, sensor devices, etc. With theimplementation solutions provided in the present specification, an imagecapture device or a video shooting device of a client can be used tosend image data of goods detection to a server, so that the serverperforms processes such as goods identification and order settlement.Therefore, overall hardware implementation costs can be lower. Inaddition, data exchanging nodes can be reduced and solutionimplementations can be simpler and more convenient. In actualapplications, implementation costs of the technical solutions forautomatic settlement processing of self-service supermarket goods orderscan be greatly reduced, and the technical solutions are convenient toimplement and have better stability. An implementation environment andimplementation factors such as a self-service supermarket size, acustomer flow volume, and a consumer behavior feature have relativelysmall impact, and facilitate large-scale promotion and use.

The goods order processing method or apparatus provided in theimplementations of the present specification can be implemented by aprocessor executing a corresponding program instruction in a computer.For example, it can be implemented at a PC end by using the C++ languagein a Windows operating system, or implemented by using a correspondingprogram design language in another system such as Linux, Android, orIOS, or implemented based on processing logic of a quantum computer. Theapparatus can be applied to a plurality of servers such as a pluralityof types of order settlement system platforms, self-service shoppingorder processing systems, and shopping cloud service system, toimplement rapid, highly-efficient, and reliable user goods orderprocessing with low costs. The present specification provides a server.As shown in FIG. 22, the server can include at least one processor and amemory configured to store an instruction executable by the processor.When executing the instruction, the processor implements the following:receiving an associated image uploaded by a client, where the associatedimage includes an image obtained when the client detects, based on animage recognition method, that a change of the goods occurs in a storagecontainer; identifying information about the goods difference in thestorage container based on the associated image; generatingcorresponding goods order change information by using the differenceinformation; updating user order information corresponding to the clientbased on the goods order change information; and sending updated userorder information to the client.

Notably, the previous server can further include another implementationmethod based on the descriptions of the method or apparatusimplementations. For a specific implementation method, reference can bemade to the descriptions in the related method implementations, anddetails are omitted here.

The server can include a separate server, or can include a server systemcomposed of a plurality of servers, for example, a cloud server forstoring user orders, a server for manual operation task scheduling, aserver operated by a network cashier, a server for automatic video orimage analysis, etc. or can be a distributed server or a server clusterarchitecture.

Notably, the previous apparatus or electronic device in the presentspecification can further include another implementation method based onthe descriptions in the related method implementations. For a specificimplementation method, reference can be made to the descriptions in themethod implementations, and details are omitted here. Theimplementations in the present specification are described in aprogressive way. For same or similar parts in the implementations,reference can be made to the implementations. Each implementationfocuses on a difference from other implementations. Particularly, ahardware and program implementation is basically similar to a methodimplementation, and therefore is described briefly; for related parts,reference can be made to partial descriptions in the methodimplementations.

Certainly, based on the descriptions of the method or apparatusimplementations on the client side, the present specification canfurther provide a shopping terminal. The shopping terminal can includethe previous mobile phone with, for example, photographing and imagedata transmission functions, or can further include end-user devicessuch as a shopping cart and a positioning device. The client can includean image capture unit and a detection unit, and can be implemented by aprocessor executing a corresponding program instruction. For example, itcan be implemented at a PC end by using the C++ language in a Windowsoperating system, or implemented by using a program design language inanother system such as Linux, Android, or IOS, or implemented based onprocessing logic of a quantum computer. The shopping terminal cancapture an image of a shopping cart and another storage container. Whena consumer puts in or takes out goods, a change of the goods can bedetected, and the captured image can be uploaded to the server, so thatthe server can identify goods in the image, to generate or process userorder information and further implement rapid, highly-efficient, andreliable user goods order processing with low costs. The presentspecification provides a client. The client can include at least oneprocessor and a memory configured to store an instruction executable bythe processor. When executing the instruction, the processor implementsthe following: detecting, based on an image recognition method, whethera change of the goods occurs in a storage container; if a change of thegoods is detected, sending an associated image of the change of thegoods in the storage container to a server; receiving updated user orderinformation returned by the server; and displaying the updated userorder information.

Notably, the previous client can further include another implementationmethod based on the descriptions of the method or apparatusimplementations. For a specific implementation method, reference can bemade to the descriptions in the related method implementations, anddetails are omitted here.

Specific implementations of the present specification are describedabove. Other implementations fall within the scope of the appendedclaims. In some situations, the actions or steps described in the claimscan be performed in a sequence different from the sequence in theimplementation and the desired results can still be achieved. Inaddition, the process depicted in the accompanying drawings does notnecessarily need a particular execution sequence to achieve the desiredresults. In some implementations, multi-tasking and parallel processingcan be advantageous.

Based on the descriptions of the method or the apparatus or the clientor the server, the present specification further provides one or moregoods order processing systems. FIG. 23 is a schematic architecturaldiagram illustrating an implementation of the system, according to oneimplementation of the present specification. The system can includeshopping terminal 1 and server 2.

Server 1 can include the apparatus according to any of theimplementations on the server side or can implement steps of any methodon the server side.

Shopping terminal 2 can include the shopping terminal according to anyone of the implementations or can implement steps of any one of themethods on the shopping terminal side.

According to the goods order processing method, apparatus, and system,the server, and the shopping terminal that are provided in one or moreimplementations of the present specification, a photographing device canbe installed in a shopping basket, a shopping cart, or other storagecontainers, and a remote server can identify and determine a user orderchange by monitoring a change of the goods in the storage container inreal time, and can generate or update a shopping list of a user. Assuch, end-user devices such as an intelligent shopping cart can be usedto process data of self-service supermarket orders in combination with aserver for goods identification and order processing to reduceself-service cashier devices, sensor devices, etc. With theimplementation solutions provided in the present specification, an imagecapture device or a video shooting device of a client can be used tosend image data of goods detection to a server, so that the serverperforms processes such as goods identification and order settlement.Therefore, overall hardware implementation costs can be lower. Inaddition, data exchanging nodes can be reduced and solutionimplementations can be simpler and more convenient. In actualapplications, implementation costs of the technical solutions forautomatic settlement processing of self-service supermarket goods orderscan be greatly reduced, and the technical solutions are convenient toimplement and have better stability. An implementation environment andimplementation factors such as a self-service supermarket size, acustomer flow volume, and a consumer behavior feature have relativelysmall impact, and facilitate large-scale promotion and use.

The content of the implementations of the present specification providesdata description, storage, acquisition, interaction, calculation,determining, etc. of, for example, image capture and transmission byusing a smartphone, detection model construction by using an RCNNmethod, cashier operation status setting for task allocation, and aplurality of processing methods upon an in-image-goods identificationfailure. However, the implementations of the present specification arenot limited to satisfy an industry communications standard, a machinelearning model, standard computer data processing and a data storagerule, or situations described in the one or more implementations of thepresent specification. A slightly modified implementation solutionobtained by using some industry standards or a machine learning model,or in a self-defined way, or on a basis of the described implementationscan also implement an implementation effect that is the same as,equivalent to, or similar to the described implementation, or anexpected implementation effect obtained after transformation. Theimplementations of obtaining, storing, determining, processing, etc. ofmodified or deformed data can still fall within the scope of theoptional implementation solutions of the implementations of the presentspecification.

Specific implementations of the present specification are describedabove. Other implementations fall within the scope of the appendedclaims. In some situations, the actions or steps described in the claimscan be performed in a sequence different from the sequence in theimplementation and the desired results can still be achieved. Inaddition, the process depicted in the accompanying drawings does notnecessarily need a particular execution sequence to achieve the desiredresults. In some implementations, multi-tasking and parallel processingcan be advantageous.

In the 1990s, whether a technology improvement is a hardware improvement(for example, an improvement of a circuit structure, such as a diode, atransistor, or a switch) or a software improvement (an improvement of amethod procedure) can be obviously distinguished. However, astechnologies develop, current improvements for many method procedurescan be considered as direct improvements of hardware circuit structures.A designer usually programs an improved method procedure into a hardwarecircuit, to obtain a corresponding hardware circuit structure.Therefore, a method procedure can be improved by using a hardware entitymodule. For example, a programmable logic device (PLD) (for example, afield programmable gate array (FPGA)) is such an integrated circuit, anda logical function of the PLD is determined by a user through deviceprogramming. The designer performs programming to “integrate” a digitalsystem to a PLD without requesting a chip manufacturer to design andproduce a dedicated integrated circuit chip. In addition, at present,instead of manually manufacturing an integrated circuit chip, suchprogramming is mostly implemented by using “logic compiler” software.The logic compiler is similar to a software compiler used to develop andwrite a program. Original code needs to be written in a particularprogramming language for compilation. The language is referred to as ahardware description language (HDL). There are many HDLs, such as theAdvanced Boolean Expression Language (ABEL), the Altera HardwareDescription Language (AHDL), Confluence, the Cornell UniversityProgramming Language (CUPL), HDCal, the Java Hardware DescriptionLanguage (JHDL), Lava, Lola, MyHDL, PALASM, and the Ruby HardwareDescription Language (RHDL). The very-high-speed integrated circuithardware description language (VHDL) and Verilog2 are most commonlyused. A person skilled in the art should also understand that a hardwarecircuit that implements a logical method procedure can be readilyobtained once the method procedure is logically programmed by using theseveral described hardware description languages and is programmed intoan integrated circuit.

A controller can be implemented by using any appropriate methods. Forexample, the controller can be a microprocessor or a processor, or acomputer-readable medium that stores computer readable program code(such as software or firmware) that can be executed by themicroprocessor or the processor, a logic gate, a switch, anapplication-specific integrated circuit (ASIC), a programmable logiccontroller, or an embedded microprocessor. Examples of the controllerinclude but are not limited to the following microprocessors: ARC 625D,Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. Thememory controller can also be implemented as a part of the control logicof the memory. A person skilled in the art also knows that, in additionto implementing the controller by using the computer readable programcode, logic programming can be performed on method steps to allow thecontroller to implement the same function in forms of the logic gate,the switch, the application-specific integrated circuit, theprogrammable logic controller, and the embedded microcontroller.Therefore, the controller can be considered as a hardware component, andan apparatus included in the controller and configured to implementvarious functions can also be considered as a structure in the hardwarecomponent. Or, the apparatus configured to implement various functionscan even be considered as both a software module implementing the methodand a structure in the hardware component.

The system, apparatus, module, or unit illustrated in the previousimplementations can be implemented by using a computer chip or anentity, or can be implemented by using a product having a certainfunction. A typical implementation device is a computer. Specifically,the computer can be, for example, a personal computer, a laptopcomputer, a vehicle-mounted human computer interaction device, acellular phone, a digital camera phone, a smartphone, a personal digitalassistant, a media player, a navigation device, an email device, a gameconsole, a tablet computer, or a wearable device, or a combination ofany of these devices.

Although the one or more implementations of the present specificationprovide the operation steps of the method according to an implementationor a flowchart, the conventional or non-creative means can include moreor fewer operation steps. The sequence of the steps listed in theimplementations is merely one of numerous step execution sequences, anddoes not represent the unique execution sequence. In actual execution ofan apparatus or a terminal product, execution can be performed based ona method sequence shown in the implementations or the accompanyingdrawings, or performed in parallel (for example, a parallel processor ora multi-threaded processing environment, or even a distributed dataprocessing environment). The terms “include”, “include”, or their anyother variant is intended to cover a non-exclusive inclusion, so that aprocess, a method, a product, or a device that includes a list ofelements not only includes those elements but also includes otherelements which are not expressly listed, or further includes elementsinherent to such process, method, product, or device. An elementpreceded by “includes a . . . ” does not, without more constraints,preclude the existence of additional identical elements in the process,method, product, or device that includes the element.

For ease of description, the previous apparatus is described by dividingthe functions into various modules. Certainly, when the one or moreimplementations of the present specification are implemented, thefunctions of each module can be implemented in one or more pieces ofsoftware and/or hardware, or a module implementing a same function canbe implemented by a combination of a plurality of submodules orsubunits. The described apparatus implementation is merely an example.For example, the unit division is merely logical function division andcan be other division in actual implementation. For example, a pluralityof units or components can be combined or integrated into anothersystem, or some features can be ignored or not performed. In addition,the displayed or discussed mutual couplings or direct couplings orcommunication connections can be implemented by using some interfaces.The indirect couplings or communication connections between theapparatuses or units can be implemented in electronic, mechanical, orother forms.

A person skilled in the art also knows that, in addition to implementingthe controller by using the computer readable program code, logicprogramming can be performed on method steps to allow the controller toimplement the same function in forms of the logic gate, the switch, theapplication-specific integrated circuit, the programmable logiccontroller, and the embedded microcontroller. Therefore, the controllercan be considered as a hardware component, and an apparatus included inthe controller and configured to implement various functions can also beconsidered as a structure in the hardware component. Or, the apparatusconfigured to implement various functions can even be considered as botha software module implementing the method and a structure in thehardware component.

The present disclosure is described with reference to the flowchartsand/or block diagrams of the method, the device (system), and thecomputer program product according to the implementations of the presentdisclosure. It should be understood that computer program instructionscan be used to implement each process and/or each block in theflowcharts and/or the block diagrams and a combination of a processand/or a block in the flowcharts and/or the block diagrams. Thesecomputer program instructions can be provided for a general-purposecomputer, a dedicated computer, an embedded processor, or a processor ofany other programmable data processing device to generate a machine, sothat the instructions executed by a computer or a processor of any otherprogrammable data processing device generate an apparatus forimplementing a specific function in one or more processes in theflowcharts and/or in one or more blocks in the block diagrams.

These computer program instructions can be stored in a computer readablememory that can instruct the computer or any other programmable dataprocessing devices to work in a specific manner, so that theinstructions stored in the computer readable memory generate an artifactthat includes an instruction apparatus. The instruction apparatusimplements a specific function in one or more processes in theflowcharts and/or in one or more blocks in the block diagrams.

These computer program instructions can also be loaded onto a computeror another programmable data processing device, so that a series ofoperations and steps are performed on the computer or the anotherprogrammable device to generate computer-implemented processing.Therefore, the instructions executed on the computer or otherprogrammable devices provide steps for implementing a specific functionin one or more processes in the flowcharts and/or in one or more blocksin the block diagrams.

In a typical configuration, a computing device includes one or moreprocessors (CPU), an input/output interface, a network interface, and amemory.

The memory can possibly include a non-persistent memory, a random accessmemory (RAM), a non-volatile memory, and/or another form in a computerreadable medium, for example, a read-only memory (ROM) or a flash memory(flash RAM). The memory is an example of the computer readable medium.

The computer readable medium includes persistent, non-persistent,movable, and unmovable media that can implement information storage byusing any method or technology. Information can be a computer readableinstruction, a data structure, a program module, or other data. Anexample of a computer storage medium includes but is not limited to aphase change random access memory (PRAM), a static random access memory(SRAM), a dynamic random access memory (DRAM), another type of randomaccess memory (RAM), a read-only memory (ROM), an electrically erasableprogrammable read-only memory (EEPROM), a flash memory or another memorytechnology, a compact disc read-only memory (CD-ROM), a digitalversatile disc (DVD) or another optical storage, a cassette magnetictape, a tape and disk storage, a graphene memory, or another magneticstorage device or any other non-transmission media that can beconfigured to store information that a computing device can access.Based on the description in the present specification, the computerreadable medium does not include transitory computer-readable media(transitory media), for example, a modulated data signal and carrier.

A person skilled in the art should understand that the one or moreimplementations of the present specification can be provided as amethod, a system, or a computer program product. Therefore, the one ormore implementations of the present specification can use a form ofhardware only implementations, software only implementations, orimplementations with a combination of software and hardware. Inaddition, the one or more implementations of the present specificationcan use a form of a computer program product that is implemented on oneor more computer-usable storage media (including but not limited to adisk memory, a CD-ROM, an optical memory, etc.) that includecomputer-usable program code.

The one or more implementations of the present specification can bedescribed in common contexts of computer executable instructionsexecuted by a computer, such as a program module. Generally, the programmodule includes a routine, a program, an object, a component, a datastructure, etc. executing a specific task or implementing a specificabstract data type. The one or more implementations of the presentspecification can also be practiced in distributed computingenvironments. In these distributed computing environments, tasks areexecuted by remote processing devices that are connected by using acommunications network. In a distributed computing environment, theprogram module can be located in both local and remote computer storagemedia including storage devices.

The implementations in the present specification are described in aprogressive way. Same or similar parts in the implementations canreference to each other. Each implementation focuses on a differencefrom other implementations. Particularly, a system implementation isbasically similar to a method implementation, and therefore, isdescribed briefly. For related parts, reference can be made to relateddescriptions in the method implementation. In descriptions in thepresent specification, descriptions about such reference terms as “animplementation”, “some implementations”, “an example”, “a specificexample”, and “some examples” mean that specific features, structures,materials, or characteristics described with reference to theimplementations or examples are included in at least one implementationor example of the present specification. In the present specification,the previous example expressions of the terms are not necessarily withrespect to a same implementation or example. In addition, the describedspecific features, structures, materials, or characteristics can becombined in a proper way in any one or more of the implementations orexamples. In addition, a person skilled in the art can integrate orcombine different implementations or examples and characteristics ofdifferent implementations or examples described in the presentspecification, provided that they do not conflict with each other.

The previous descriptions are merely implementations of the one or moreimplementations of the present specification, and are not intended tolimit the one or more implementations of the present specification. Fora person skilled in the art, the one or more implementations of thepresent specification can have various modifications and changes. Anymodifications, equivalent replacements, improvements, etc. made withinthe spirit and principle of the present application shall fall withinthe protection scope of the claims.

Embodiments and the operations described in this specification can beimplemented in digital electronic circuitry, or in computer software,firmware, or hardware, including the structures disclosed in thisspecification or in combinations of one or more of them. The operationscan be implemented as operations performed by a data processingapparatus on data stored on one or more computer-readable storagedevices or received from other sources. A data processing apparatus,computer, or computing device may encompass apparatus, devices, andmachines for processing data, including by way of example a programmableprocessor, a computer, a system on a chip, or multiple ones, orcombinations, of the foregoing. The apparatus can include specialpurpose logic circuitry, for example, a central processing unit (CPU), afield programmable gate array (FPGA) or an application-specificintegrated circuit (ASIC). The apparatus can also include code thatcreates an execution environment for the computer program in question,for example, code that constitutes processor firmware, a protocol stack,a database management system, an operating system (for example anoperating system or a combination of operating systems), across-platform runtime environment, a virtual machine, or a combinationof one or more of them. The apparatus and execution environment canrealize various different computing model infrastructures, such as webservices, distributed computing and grid computing infrastructures.

A computer program (also known, for example, as a program, software,software application, software module, software unit, script, or code)can be written in any form of programming language, including compiledor interpreted languages, declarative or procedural languages, and itcan be deployed in any form, including as a stand-alone program or as amodule, component, subroutine, object, or other unit suitable for use ina computing environment. A program can be stored in a portion of a filethat holds other programs or data (for example, one or more scriptsstored in a markup language document), in a single file dedicated to theprogram in question, or in multiple coordinated files (for example,files that store one or more modules, sub-programs, or portions ofcode). A computer program can be executed on one computer or on multiplecomputers that are located at one site or distributed across multiplesites and interconnected by a communication network.

Processors for execution of a computer program include, by way ofexample, both general- and special-purpose microprocessors, and any oneor more processors of any kind of digital computer. Generally, aprocessor will receive instructions and data from a read-only memory ora random-access memory or both. The essential elements of a computer area processor for performing actions in accordance with instructions andone or more memory devices for storing instructions and data. Generally,a computer will also include, or be operatively coupled to receive datafrom or transfer data to, or both, one or more mass storage devices forstoring data. A computer can be embedded in another device, for example,a mobile device, a personal digital assistant (PDA), a game console, aGlobal Positioning System (GPS) receiver, or a portable storage device.Devices suitable for storing computer program instructions and datainclude non-volatile memory, media and memory devices, including, by wayof example, semiconductor memory devices, magnetic disks, andmagneto-optical disks. The processor and the memory can be supplementedby, or incorporated in, special-purpose logic circuitry.

Mobile devices can include handsets, user equipment (UE), mobiletelephones (for example, smartphones), tablets, wearable devices (forexample, smart watches and smart eyeglasses), implanted devices withinthe human body (for example, biosensors, cochlear implants), or othertypes of mobile devices. The mobile devices can communicate wirelessly(for example, using radio frequency (RF) signals) to variouscommunication networks (described below). The mobile devices can includesensors for determining characteristics of the mobile device's currentenvironment. The sensors can include cameras, microphones, proximitysensors, GPS sensors, motion sensors, accelerometers, ambient lightsensors, moisture sensors, gyroscopes, compasses, barometers,fingerprint sensors, facial recognition systems, RF sensors (forexample, Wi-Fi and cellular radios), thermal sensors, or other types ofsensors. For example, the cameras can include a forward- or rear-facingcamera with movable or fixed lenses, a flash, an image sensor, and animage processor. The camera can be a megapixel camera capable ofcapturing details for facial and/or iris recognition. The camera alongwith a data processor and authentication information stored in memory oraccessed remotely can form a facial recognition system. The facialrecognition system or one-or-more sensors, for example, microphones,motion sensors, accelerometers, GPS sensors, or RF sensors, can be usedfor user authentication.

To provide for interaction with a user, embodiments can be implementedon a computer having a display device and an input device, for example,a liquid crystal display (LCD) or organic light-emitting diode(OLED)/virtual-reality (VR)/augmented-reality (AR) display fordisplaying information to the user and a touchscreen, keyboard, and apointing device by which the user can provide input to the computer.Other kinds of devices can be used to provide for interaction with auser as well; for example, feedback provided to the user can be any formof sensory feedback, for example, visual feedback, auditory feedback, ortactile feedback; and input from the user can be received in any form,including acoustic, speech, or tactile input. In addition, a computercan interact with a user by sending documents to and receiving documentsfrom a device that is used by the user; for example, by sending webpages to a web browser on a user's client device in response to requestsreceived from the web browser.

Embodiments can be implemented using computing devices interconnected byany form or medium of wireline or wireless digital data communication(or combination thereof), for example, a communication network. Examplesof interconnected devices are a client and a server generally remotefrom each other that typically interact through a communication network.A client, for example, a mobile device, can carry out transactionsitself, with a server, or through a server, for example, performing buy,sell, pay, give, send, or loan transactions, or authorizing the same.Such transactions may be in real time such that an action and a responseare temporally proximate; for example an individual perceives the actionand the response occurring substantially simultaneously, the timedifference for a response following the individual's action is less than1 millisecond (ms) or less than 1 second (s), or the response is withoutintentional delay taking into account processing limitations of thesystem.

Examples of communication networks include a local area network (LAN), aradio access network (RAN), a metropolitan area network (MAN), and awide area network (WAN). The communication network can include all or aportion of the Internet, another communication network, or a combinationof communication networks. Information can be transmitted on thecommunication network according to various protocols and standards,including Long Term Evolution (LTE), 5G, IEEE 802, Internet Protocol(IP), or other protocols or combinations of protocols. The communicationnetwork can transmit voice, video, biometric, or authentication data, orother information between the connected computing devices.

Features described as separate implementations may be implemented, incombination, in a single implementation, while features described as asingle implementation may be implemented in multiple implementations,separately, or in any suitable sub-combination. Operations described andclaimed in a particular order should not be understood as requiring thatthe particular order, nor that all illustrated operations must beperformed (some operations can be optional). As appropriate,multitasking or parallel-processing (or a combination of multitaskingand parallel-processing) can be performed.

1. A computer-implemented method for processing an order of goods,comprising: receiving at least one associated image uploaded by aclient, wherein the at least one associated image is obtained inresponse to detecting, by the client based on an image recognitionmethod, an occurrence of a change of goods in a storage container;identifying, based on the at least one image, difference informationpertaining to a difference in the goods in the storage container;generating, using the difference information, corresponding goods orderchange information; updating user order information corresponding to theclient based on the goods order change information; and sending updateduser order information to the client.
 2. The computer-implemented methodaccording to claim 1, wherein the at least one associated imagecomprises at least one of the following images: at least one first goodsimage corresponding to difference data that satisfies a changethreshold; at least one second goods image captured within apredetermined window corresponding to a moment of the change of thegoods, the at least one second goods image being captured in response todetecting the occurrence of the change of the goods in the storagecontainer; and at least one third goods image captured in apredetermined time range corresponding to the moment of the change ofthe goods, the at least one third good image being captured in responseto detecting the occurrence of the change of the goods in the storagecontainer.
 3. The computer-implemented method according to claim 1,wherein identifying the difference information pertaining to thedifference in the goods in the storage container based on the at leastone associated image comprises: obtaining the at least one associatedimage; and detecting goods information in the at least one associatedimage by using a goods detection model obtained through a machinelearning algorithm, to determine the difference information.
 4. Thecomputer-implemented method according to claim 1, wherein identifyingthe difference information pertaining to the difference in the goods inthe storage container based on the at least one associated imagecomprises: detecting a goods identifier in the at least one associatedimage; using the goods identifier as the difference information; and inresponse to determining that the goods identifier in the at least oneassociated image fails to be identified, detecting goods information inthe at least one associated image by using a goods detection modelobtained through a machine learning algorithm to determine thedifference information.
 5. The computer-implemented method according toclaim 4, further comprising: in response to determining that a goods inthe at least one associated image fails to be identified, generating anoperation task for a detection failure of the at least one associatedimage; placing the operation task in a task scheduling queue; andpushing, based on a network cashier list, the operation task in the taskscheduling queue to cashier nodes that satisfy an operation condition,wherein the network cashier list records task operation statuses ofcashier nodes.
 6. The computer-implemented method according to claim 5,further comprising: identifying a network cashier in the network cashierlist bound to the operation task; and sending the operation task to acashier node of the network cashier for processing.
 7. Thecomputer-implemented method according to claim 6, further comprising,after identifying the network cashier in the network cashier list boundto the operation task: querying an operation status of the cashier node;and in response to determining that the operation status indicates asaturated operation amount, sending the operation task to a cashier nodeof a network cashier with an unsaturated operation amount in the networkcashier list for processing.
 8. The computer-implemented methodaccording to claim 5, wherein pushing, based on the network cashierlist, the operation task in the task scheduling queue to the cashiernodes that satisfy the operation condition comprises: pushing theoperation task to at least two cashier nodes; and wherein identifying,based on the at least one image, the difference information pertainingto the difference in the goods in the storage container comprises:determining the difference information pertaining to the difference inthe goods in the storage container based on goods identification resultsof the at least two cashier nodes.
 9. The computer-implemented methodaccording to claim 5, further comprising: in response to determiningthat a goods identification result of a cashier node comprises a goodsidentification failure, performing at least one of the following:returning a prompt message to the client, wherein the prompt messagecomprises information content of at least one of a goods ordergeneration failure and a goods re-arrangement request; and generating anunidentified-goods order corresponding to an associated image for thegoods identification failure, and setting, based on information aboutthe unidentified-goods order, an identity verification result of astorage container corresponding to the unidentified-goods order to“failed”.
 10. The computer-implemented method according to claim 1,further comprising: receiving positioning information of the storagecontainer uploaded by the client; determining a goods interval for thechange of the goods in the storage container based on the positioninginformation of the storage container; and wherein identifying thedifference information pertaining to the difference in the goods in thestorage container based on the at least one associated image comprises:identifying goods information in the at least one associated image in arange of the goods interval; and determining the difference informationfor the change of the goods in the storage container based on the goodsinformation.
 11. A non-transitory, computer-readable medium storing oneor more instructions executable by a computer system to performoperations comprising: receiving at least one associated image uploadedby a client, wherein the at least one associated image is obtained inresponse to detecting, by the client based on an image recognitionmethod, an occurrence of a change of goods in a storage container;identifying, based on the at least one image, difference informationpertaining to a difference in the goods in the storage container;generating, using the difference information, corresponding goods orderchange information; updating user order information corresponding to theclient based on the goods order change information; and sending updateduser order information to the client.
 12. The non-transitory,computer-readable medium according to claim 11, wherein the at least oneassociated image comprises at least one of the following images: atleast one first goods image corresponding to difference data thatsatisfies a change threshold; at least one second goods image capturedwithin a predetermined window corresponding to a moment of the change ofthe goods, the at least one second goods image being captured inresponse to detecting the occurrence of the change of the goods in thestorage container; and at least one third goods image captured in apredetermined time range corresponding to the moment of the change ofthe goods, the at least one third good image being captured in responseto detecting the occurrence of the change of the goods in the storagecontainer.
 13. The non-transitory, computer-readable medium according toclaim 11, wherein identifying the difference information pertaining tothe difference in the goods in the storage container based on the atleast one associated image comprises: obtaining the at least oneassociated image; and detecting goods information in the at least oneassociated image by using a goods detection model obtained through amachine learning algorithm, to determine the difference information. 14.The non-transitory, computer-readable medium according to claim 11,wherein identifying the difference information pertaining to thedifference in the goods in the storage container based on the at leastone associated image comprises: detecting a goods identifier in the atleast one associated image; using the goods identifier as the differenceinformation; and in response to determining that the goods identifier inthe at least one associated image fails to be identified, detectinggoods information in the at least one associated image by using a goodsdetection model obtained through a machine learning algorithm todetermine the difference information.
 15. The non-transitory,computer-readable medium according to claim 14, further comprising: inresponse to determining that a goods in the at least one associatedimage fails to be identified, generating an operation task for adetection failure of the at least one associated image; placing theoperation task in a task scheduling queue; and pushing, based on anetwork cashier list, the operation task in the task scheduling queue tocashier nodes that satisfy an operation condition, wherein the networkcashier list records task operation statuses of cashier nodes.
 16. Thenon-transitory, computer-readable medium according to claim 15, furthercomprising: identifying a network cashier in the network cashier listbound to the operation task; and sending the operation task to a cashiernode of the network cashier for processing.
 17. The non-transitory,computer-readable medium according to claim 16, further comprising,after identifying the network cashier in the network cashier list boundto the operation task: querying an operation status of the cashier node;and in response to determining that the operation status indicates asaturated operation amount, sending the operation task to a cashier nodeof a network cashier with an unsaturated operation amount in the networkcashier list for processing.
 18. The non-transitory, computer-readablemedium according to claim 15, wherein pushing, based on the networkcashier list, the operation task in the task scheduling queue to thecashier nodes that satisfy the operation condition comprises: pushingthe operation task to at least two cashier nodes; and whereinidentifying, based on the at least one image, the difference informationpertaining to the difference in the goods in the storage containercomprises: determining the difference information pertaining to thedifference in the goods in the storage container based on goodsidentification results of the at least two cashier nodes.
 19. Thenon-transitory, computer-readable medium according to claim 15, furthercomprising: in response to determining that a goods identificationresult of a cashier node comprises a goods identification failure,performing at least one of the following: returning a prompt message tothe client, wherein the prompt message comprises information content ofat least one of a goods order generation failure and a goodsre-arrangement request; and generating an unidentified-goods ordercorresponding to an associated image for the goods identificationfailure, and setting, based on information about the unidentified-goodsorder, an identity verification result of a storage containercorresponding to the unidentified-goods order to “failed”.
 20. Acomputer-implemented system, comprising: one or more computers; and oneor more computer memory devices interoperably coupled with the one ormore computers and having tangible, non-transitory, machine-readablemedia storing one or more instructions that, when executed by the one ormore computers, perform one or more operations comprising: receiving atleast one associated image uploaded by a client, wherein the at leastone associated image is obtained in response to detecting, by the clientbased on an image recognition method, an occurrence of a change of goodsin a storage container; identifying, based on the at least one image,difference information pertaining to a difference in the goods in thestorage container; generating, using the difference information,corresponding goods order change information; updating user orderinformation corresponding to the client based on the goods order changeinformation; and sending updated user order information to the client.21-44. (canceled)